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Abstract 

Large ensembles of globally coupled chaotic neural networks undergo a transition 
to complete synchronization for high coupling intensities. The onset of this fully 
coherent behavior is preceded by a regime where clusters of networks with identical 
activity are spontaneously formed. In these regimes of coherent collective evolution 
the dynamics of each neural network is still chaotic. These results may be relevant 
to the study of systems where interaction between elements is able to give rise to 
coherent complex behavior, such as in cortex activity or in insect societies. 
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Mutual synchronization may play a significant role in the emergence of coherent 
collective behavior of different biological populations [1] . Studies of globally coupled 
nonlinear oscillators show the possibility of spontaneous transitions to a coherent 
regime characterized by identical states of all elements [2-5]. Similar transitions, 
known as complete stochastic synchronization, have been found for globally coupled 
chaotic logistic maps [6], employed in the description of ecological dynamics [7], 
and for populations of several coupled chaotic neurons [8]. However, they deem im- 
probable when larger and more complex systems are considered. In contrast to this 
view, our study reveals that complete mutual synchronization can easily be achieved 
even in ensembles whose members represent large neural networks with complex in- 
dividual dynamics. In the emerging coherent regime, the signals generated by all 
networks in the ensemble are identical. The transition is preceded by clustering, 
where groups of neural networks with coherent activity patterns arc spontaneously 
formed. Our finding may contribute towards better understanding of coherent ac- 
tivity in the cortex and in biological populations, such as swarms or insect societies. 
It can find practical applications in the design of robotic ensembles. 

Real neurons are comphcated and, in theoretical studies of collective neural 
activity, greatly simplified models are used [9-11]. To investigate synchronization in 
ensembles of coupled neural networks, we employ the traditional model of McCuUoch 
and Pitts [12] where each neuron is specified by its activity that changes in response 
to signals arriving from other neurons. A network consists of a set of such elements 
linked through activatory or inhibitory connections. The dynamics of such a network 
with arbitrary asymmetric connections is typically characterized by an irregular 
sequence of complex activity patterns. 

In this Letter we consider ensembles made of N identical neural networks each 
consisting of K neurons. The collective dynamics of an ensemble is described by the 
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following algorithm: At time t + 1, the activity x\ of a neuron k — 1,...,K belonging 
to a network i — 1, is 



where h\ = J2iLi Jkix'i{t) is the signal arriving to this neuron at time t from all 
other elements of the same network, Jki are the connection weights (the same for 
all networks), and ©(z) is a sigmoidal function, such as ©(z) = for 2; < and 
Q{z) = 1 for z > 0. 

The two terms in the right side of Eq. (1) have a clear interpretation. The first of 
them represents the individual response of a neuron to the total signal received from 
other elements in its own network. The second term depends on the global signal 
obtained by summation of individual signals received by neurons occupying the same 
positions in all networks of the ensemble. The parameter e specifies the strength 
of global couphng. When global coupling is absent (e = 0), the networks forming 
the ensemble are independent. On the other hand, at £ = 1 the first term vanishes 
and the states of respective neurons in all networks must be identical since they 
are determined by the same global signal. For < £ < 1, the ensemble dynamics 
is governed by an interplay between local coupling inside the networks and global 
coupling across them. 

Below we show that the model exhibits, under increasing the global coupling 
intensity, a spontaneous transition to a coherent collective behavior. This transi- 
tion is characterized by formation of coherent network clusters followed by complete 
synchronization of all networks in the ensemble. It takes place already at low inten- 
sities of global coupling and is observed under an arbitrary choice of the connection 
weights in elementary networks. 

Our analysis is based on numerical investigations. As the first step, we have 
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set up the connection weights between neurons in the individual network. Each 
of the connection weights Jki between neurons has been chosen at random with 
equal probability from the interval between —1 to 1. The weights of forward and 
reverse connections were independently selected, and therefore Jki ^ Jik- Most 
of the simulations were performed for ensembles of A/" = 100 identical networks, 
each consisting of X = 50 neurons. The connection weights remained fixed within 
the entire series of simulations with varying global coupling intensity. The initial 
conditions for all neurons in all networks in each simulation have been randomly 
chosen. 

Since subsequent states of all neurons in all networks are recorded, each sim- 
ulation yields a large volume of data that should be further analyzed in order to 
detect coherence in the collective activity of the ensemble. An important property is 
the integral time-dependent activity Ui{t) = J2k=i ^U^) each network i = 1, ...,N 
in the ensemble. If global coupling is absent or very weak, the networks are in- 
dependent and, since the initial conditions are various for different networks, their 
activity patterns are not correlated. Therefore, the integral signals Ui{t) generated 
by different networks in the ensemble would be asynchronous. 

Figure 1 shows typical integral signals generated by networks at higher intensities 
of global coupling. The initial conditions are chosen at random for each of the 
networks and therefore the integral signals of the networks are at first not correlated. 
However, starting from a certain moment, some of the networks in the ensemble 
begin to generate identical (up to the computer precision) signals, indicating the 
onset of synchronization in the system. When e = 0.35 (Fig. lA), the entire 
ensemble breaks eventually down into several coherent clusters (the signals generated 
by networks from the same cluster are displayed here using the same color). At a 
higher intensity of global couphng {e — 0.5, Fig. IB), the activity of all networks in 
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the ensemble becomes eventually coherent. Quite remarkably, the coherent signals 
are still very complex and apparently chaotic. 

The degree of coherence in the ensemble dynamics can be characterized by the 
dispersion of the activity patterns, defined as D{t) — J2iLi J2k=i i^ki^) ~ ^k{t)]^ 
with Xk{t) — N~^Y.^^^xl{t). Fig. 2 shows, in logarithmic scale, how this property 
evolves in time in a typical simulation at a fixed intensity £ = 0.5 of global coupling. 
Within a transient period, the dispersion D{t) fiuctuates but remains, on the av- 
erage, approximately constant. Then, suddenly, it starts to exponentially decrease 
with time and rapidly approaches zero. When this occurs, full synchronization of 
all networks in the ensemble is achieved. 

Though the dispersion serves as a good indicator of full synchronization, it is 
not strongly sensitive to partial synchronization and formation of coherent clusters 
in the ensemble. To analyze clustering, a different statistical method has therefore 
been employed which invoved calculation of pair distances between activity patterns 
of all networks. The pair distance between the activity patterns of two networks i 

1/2 



and j is defined as dij — 



. By counting the number of network 



pairs in the whole ensemble that have at a given time the distances lying within 
subsequent equal intervals, a histogram of distribution over pair distances can be 
constructed. Fig. 3 shows these normalized histograms for several intensities of 
global coupling. 

When global coupling is weak (e = 0.15, Fig. 3A), the histogram has a sin- 
gle smooth maximum at a typical distance between the activity patterns of non- 
correlated networks. Increasing the coupling intensity, we find that above a certain 
critical point (si = 0.17) the nature of the histogram changes. Now, some pairs 
of networks in the ensemble have exactly the same activity patterns, so that the 
distance between them is zero. This corresponds to the presence of a peak at d — 
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in Fig. 3B for e — 0.28. When global coupling is further increased, the number of 
coherent pairs grows (e = 0.34, Fig. 3C). Apparently, the coherent networks are 
already organized into clusters. Indeed, several peaks are clearly seen in this his- 
togram. The peaks are located at pair distances between different coherent clusters. 
However, besides the coherent clusters the ensemble still has a number of networks 
with noncoherent activity. A slight increase of global couphng leads to the emer- 
gence of a clear cluster organization (e = 0.35, Fig. 3D). In this case, every network 
belongs to one of a few coherent clusters. As the coupling intensity grows, the num- 
ber of clusters gets smaller, until all networks belong to the same coherent group. 
Full coherence is established in the ensemble starting at £2 — 0.4 (this final regime 
is not shown in Fig. 3). 

We have repeated our simulations and statistical analysis for different random 
choices of connection weights in the networks and have observed basically the same 
sequence of changes leading to clustering and final synchronization in all studied 
cases, though the respective critical coupling intensities have been found to depend 
on the choice of connection weights. Moreover, essentially the same results have been 
obtained when ensembles consisting of larger networks of 100 neurons were studied 
and when other sigmoidal functions Q{z) in the algorythm (1) were employed. This 
brings us to a suggestion that emergence of coherent collective behavior might be a 
generic property of ensembles formed by globally coupled neural networks. 

Full mutual synchronization may play an important role in the action of network 
ensembles in the brain which are responsible for generation of complex temporal 
signals and/or determine motor functions. The results reported in this Letter can 
also be important for studies of animal swarms and robotic populations. Indeed, 
one can imagine that each of the networks controls the individual behavior of a 
different population member. Communication between the members may then lead 
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to global coupling between such networks which can, as we have shown, result in the 
emergence of perfectly synchronous behaviour or in the breakdown of the population 
into several coherently operating groups. 

The authors thank prof. B. Hess for enlightening discussions. D. H. Z. is grateful 
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Figure captions 

Fig. 1: Time-dependent integral activity of ten selected networks in an ensemble of 
A'" = 100 for different intensities of global coupling, corresponding to (A) 
clustering (e = 0.35) and (B) complete synchronization (e — 0.5). Synchronous 
signals are indicated by the same colors. 

Fig. 2: Dispersion of the activity patterns of all networks in the ensemble as function 
of time under synchronization conditions {e = 0.5). 

Fig. 3: Histograms of distributions over pair distances d between activity patterns of 
all networks in the ensemble for different intensities of global coupling corre- 
sponding to (A) asynchronous regime, e = 0.15, (B) onset of synchronization, 
e — 0.28, (C) partial clustering, e — 0.34, and (D) full clustering, e — 0.35. 
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